Papers with Reading comprehension
Comprehensive Multi-Dataset Evaluation of Reading Comprehension (D19-58)
Copied to clipboard
| Challenge: | Recent research aims to facilitate training and evaluation on several reading comprehension datasets at the same time. |
| Approach: | They propose an evaluation server that reports performance on seven diverse reading comprehension datasets and includes synthetic augmentations to test models' ability to handle out-of-domain questions. |
| Outcome: | The evaluation server performs on seven reading comprehension datasets, and collects and includes synthetic augmentations for these datasets to test models' ability to handle out-of-domain questions. |
DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphs (N19-1)
Copied to clipboard
| Challenge: | a large body of work has highlighted the brittleness of reading comprehension systems . a crowdsourced, adversarially-created, 55k-question benchmark requires a more comprehensive understanding of paragraphs . |
| Approach: | They propose a reading comprehension benchmark that requires Discrete Reasoning over the content of paragraphs. |
| Outcome: | The proposed benchmarks show that the best systems only achieve 38.4% F1 on the generalized accuracy metric, while human performance is 96%. |
Unsupervised Domain Adaptation of Language Models for Reading Comprehension (2020.lrec-1)
Copied to clipboard
| Challenge: | State-of-the-art reading comprehension models do not have general linguistic intelligence . accuracy of out-domain datasets is affected by the distribution of data . |
| Approach: | They propose to use supervised RC training data in the source domain and unlabeled passages in the target domain to adapt models. |
| Outcome: | The proposed model outperforms the model without domain adaptation with five datasets in different domains. |
Interpreting Themes from Educational Stories (2024.lrec-main)
Copied to clipboard
| Challenge: | Recent advances in machine reading comprehension (MRC) have centered on literal comprehension, referring to the surface-level understanding of content. |
| Approach: | They propose a dataset specifically designed for interpretive comprehension of educational narratives, providing corresponding well-edited theme texts. |
| Outcome: | The proposed dataset spans genres and cultural origins and includes human-annotated theme keywords with varying levels of granularity. |